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• This package represents the key lessons learned synthesized 
from hundreds of inputs across the Constellation Program 

— Constellation "Challenge" 

— Individual Program Office activities 
— Project lessons learned activities 
— Ares l-X test flight lessons learned 

• The target audience is senior management of new flagship 
programs or projects which might benefit from the do's and 
don'ts experienced by the Cx Program 

• Includes factors which influenced the Cx Program to provide 
context for the lessons learned 

• Is simply the tip of the iceberg 

— Full reports have been written or are planned, detailing multiple 
aspects of the program and its accomplishments 
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Key Lessons Learned 



• Robust vs. optimal planning - the only certainty is that the funding will not match 
the plan 

• Schedule creep & the fixed base - the law of diminishing returns 

• In-house tasks - sustaining the NASA institutional base vs. affordably supporting 
the Programs - getting from "or" to "and" 

• Flight Tests - learning by doing 

• Communications among a far-flung team - interpersonal networks and IT 
applications can improve bandwidth 

• Tailoring of D&C Standards - drinking from a fire hose 

• Tailoring process simplification - the law of unexpected consequences 

• Risk-Informed Design - risk as a commodity 

• Roles , responsibility ' & authority - a non-thermodynamic application of entropy 

• Decision making - is only as efficient as RRAs are clear & understood 

• Organization is organic - you'll never get it right, but you can make it better 
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NOTE: Italics indicate correlation with Cx SRB Lessons Learned 



LL Context - Factors influencing Cx 



These factors, individually & jointly, influenced the Cx Program in ways both 
beneficial and detrimental. This provides the context for Cx Lessons Learned 
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NASA's Exploration Roadmap 

What Is Our Timeline? 


unar (C 


Lunar Outpost Builjdu 


Exploration and Science Lunar Robotics Missions 


Space Shuttle Operations 


Space Shuttle Program 
Transition and Retirement 


Ares I and Orion Development 


Operations Capability Development 

(EVA Systems, Ground Operations, Mission Operations) 


Orion and Ares 


I Production and Operation 


Altair Lunar Lander Development 


Ares V and Earth Departui 


Surface Systems Development 
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Program Scope - Cx as a multi-decadal 

undertaking ), \ 


• Program focus on getting beyond LEO drove technical and programmatic 
decisions that were sub-optimal when viewed from a near-term (e.g. 

Phase I IOC only) perspective 

- Ares Upper Stage common bulkhead incorporated to provide ascent margin 
for the block 2 Orion's increased propellant load, and increased near-term risk 

— ESAS configuration to LEO included an Ares I First Stage derived from a four 
segment booster and SSME Upper Stage Engine 

• Configuration change to five segment First Stage and J-2X engine driven by desire to 
begin development of components for the heavy lifter (Ares V) needed for beyond- 
LEO missions 


• Long-term, strategic view influenced other Program decisions on 
organization, application of Agency technical standards (e.g. CEQATR), 
units of measure (SI), data architecture, etc. that were perplexing when 
viewed only from an IOC perspective 
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Inconsistent Funding: Plans vs. Reality 



• Through FY10, 10% in real budget cuts 

- $3.9B worth of content transferred out of 
Program budget of $17B 

- $1.3B of true budget reductions 

• Funding "notch" in FY10 departure from 
development profile 

- Smaller "notch" encountered each year as 
the FY tended to start with a CR 

- Impaired ability to buy down risks, invest 

in engineering development articles 

| — 1— ESAS A PMR09 — -2010 PBS 

12,000 

9.000 

6.000 
3,000 




FY06 

FY07 

FY08 

FY09 

FY10 

FY1 1 

FY12 

FY13 

FY14 

FY15 

FY16 

FY17 

FY18 

FY19 

FY20 




RY$ In M 

FY06 

FY07 

FY08 

FY09 

FY10 

FY11 

FY12 

FY13 

FY14 

FY15 

FY16 

FY17 

FY18 

FY19 

FY20 

Total 


ESAS 

2,333 

2,976 

3,195 

4,202 

5,034 

7,273 

7,994 

8,324 

8,558 

8,948 

9,547 

11,107 

11,422 

11,744 

12,073 

114730 


PMR09 

1707 

1779 

2514 

3085 

3454 

6085 

6346 

6991 

7145 

7856 

9145 

9983 

10294 

10582 

10854 

97820 


2010 PBS 

1707 

1779 

2514 

3085 

3454 

5524 

5444 

5376 

5570 

6153 

6161 

6170 

6178 

6186 

6195 

71496 

PMR09 vs. ESAS Delta 

(626) 

(1197) 

(681) 

(1117) 

(1580) 

(1188) 

(1648) 

(1333) 

(1413) 

(1092) 

(402) 

(1124) 

(1128) 

(1162) 

(1219) 

(16910) 

Passback vs. ESAS Delta 

(626) 

(1197) 

(681) 

(1117) 

(1580) 

(1749) 

(2550) 

(2948) 

(2988) 

(2795) 

(3386) 

(4937) 

(5244) 

(5558) 

(5878) 

(43234) 
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Inconsistent Funding: The Over-riding 
Context 


• Budget pressure influenced everything - exacerbated all the other factors 

- Efforts to hold schedule without sacrificing content despite funding cuts depleted Cx 
reserves 


• The Contractor's fixed base ramps up proportionately with the contract 
value, but cuts tend to be absorbed primarily in the variable content - 
deferring hardware buys, etc. 

- Schedule risk grows as more activities creep onto the critical path 

- Risk mitigation is deferred 


• Inadequate reserves or spending flexibility 

- Inability to fund risk mitigations 

- Inability to respond to cash flow disruptions resulting from annual CRs at the beginning 
of each FY 
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Funding below Plan: the Consequences 
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Robust vs. optimal planning - the only certainty 

is that the funding will not match the plan )! \ 


• Lesson Learned: Reality is that flagship programs (Cx, Webb Space 
Telescope, etc.) must be robustly planned (e.g. elastic) vs. optimally 
planned (e.g. inelastic), because absent a national imperative (e.g. race 
to the moon), funding will not show up as planned. 


• Recommendation: Plan for (1) CRs for first quarter of each year, & (2) 
strategically, programmatically, & technically decouple the 
Programs/Projects to the maximum extent, & (3) develop scenario(s) to 
accommodate a 5-10% funding shortfall in any given year. 
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Schedule Creep & the Fixed Base - the Law of '-W'. 

Diminishing Returns ), \ 


• Lesson Learned: When budgets were not appropriated as planned, 
schedule was slipped rather than cutting content. Schedule slippage 
caused Cx to have to cover a larger portion of the NASA HSF transitional 
fixed base longer, in addition to the fixed base portion of the acquisition 
itself, leading to a need for still more funding, leading to further schedule 
slippage - a vicious cycle. 


• Recommendation: Understand the inherent limitations of schedule 
slippage to resolve funding shortfalls since fixed cost accrual, both for the 
NASA Institution and the Contractor(s), erode the buying power of out- 
year $$. Program content warrants consideration in these instances. 
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Layers of Engagement 
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10 Healthy Centers 

Cx as an Agency-wide Flagship Program 


* Beneficial Effects 

- Engagement of non-traditional human 
spaceflight Centers in Cx allowed Cx to tap into 
key skills and unique facilities across the 
Agency. 

- Various Center teams contributed to Cx 
Program successes in tangible ways - 
engineering analysis, program integration, & 
flight hardware development . 



• Detrimental Effects 

- Different ways of doing business among 
Centers & contractors led to confused 
RRA across the Cx Program & convoluted 
the decision making process. 

- Large team size can mask needed changes 
to clarify RRA over time 

- Large flagship programs used as a sugar- 
daddy by Agency 
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In-house tasks - sustaining the NASA institutional base 
vs. affordably supporting the Programs - getting from 
"or" to "and" 



• Lesson Learned: Tasks performed in-house vs. contracted out are more 
than just a series of make-buy decisions. Although Cx established a 
process that emphasized distributing work in logical packages, the 
competing needs of 10 Healthy Centers, sheer program scope and the 
Program/Project phasing factor resulted in Center assignments that were 
not totally coherent, and muddled RRA. 

• Recommendation: Invest due diligence and look for the win-wins in in- 
house tasks. In-house tasks can be of value to the Program or Project and 
solve sustainment challenges for the Institution. Make affordability, 
institutional benefit, & coherence three strategic considerations in the 
make-buy decisions for in-house tasks. That is, save the Programs $$ & 
sustain the Institution while not mucking up RRA too badly - make sure 
the gain is worth the pain. 
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Flight Tests - Learning by Doing 




Lesson Learned: Many programmatic & technical methodologies 
required for efficient execution of Ares l-X & Pad Abort-1 were 
generically applicable to both flight tests and revealed 
opportunities to improve the Cx mainline effort. 

- Balancing of flight test objectives against resources 

- Selection and tailoring of applicable technical standards 

- Degree of relaxation of rigor consistent with uncrewed flight test 
context 

- Management of margin stackup/accumulation between disciplines 

- Establishing the test instrument suite specification 

- Demonstration of processes required during routine operations 

- Anchoring and validation of engineering models 

Recommendation: Resist the temptation to rely on ground test 
due to its apparent relative economy compared to flight test. A 
well-planned flight test provides significant return-on-investment, 
largely in dimensions other than just the test data. 
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Communications among a far-flung team - interpersonal 
networks and IT applications can improve bandwidth 







Lesson Learned: A network of Project & 
Program Integration Deputies fostered 
programmatic communication & issue 
resolution. "Community of Practice" 
technical networks fostered technical 
communication & issue resolution. 
Information technology (telecon, Webex, 
LifeSize, ICE/Windchill, etc.) facilitated 
effective performance for geographically 
dispersed Cx teams, although no 
substitute for F2F discussions. 

Recommendation: Flagship, 
geographically disperse Programs can 
benefit from interpersonal networks to 
enable cross-organizational 
communications. Likewise, affordable 
state-of-the-art IT tools facilitate 
communications across time & space, 
and should be put in place early & to a 
broad extent. 
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Mosaic of NASA Participation - the many 

faces of NASA in a flagship acquisition ) t \ 


• Beneficial Effects 

- Access to NASA knowledge base, needed skills & infrastructure 

• Acquisition roles (e.g. Program/Project Office) 

• Technical Issue resolution 

• GFE (in particular instances) 

• Unique facilities (i.e. test stands, chambers, etc.) 


• Detrimental Effects 

- Adherence to NPDs and D&C Standards drives contractors through 
burdensome "meets or exceeds" process 

- NASA roles tend to be fungible to cover gaps in institutional & workforce 
coverage Center to Center & project to project; lack of coherence across the 
Agency complicates Program management 

• Excessive usage of personnel in "insight" role burdens contractors 

— Technical authority vs. Program/Project authority confounds the decision 
making process 
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Tailoring of D&C Standards - 
a fire hose 


Lesson Learned: The sheer quantity of D&C standard 
imbedded requirements to be tailored is an 
impediment to any effective process. 

Recommendation: Major Aerospace Contractors build 
systems for multiple Government customers including 
DoD, NSA, & NASA among others, and not all Agencies 
require detailed standards because suppliers have 
certified internal processes & procedures that meet 
National Standards. NASA should establish an 
experienced team to audit supplier internal processes 
at the document level with a process that does not 
burden the Contractor with defending each individual 
requirement's adequacy. 

Relevant Activity: This topic is the subject of 
Affordability/Innovation Discussions with Chief 
Engineers, Health & Medical, and S&MA Leadership, 
which tasked a sub-team to describe a process and 
timeline for transforming approach to TA requirements 
a n d sta n d a rd s . Recommend this initiative establish 
clear criteria for mandatory ("shall") requirements and 
revalidate the Agency's current requirement base with 
the goal of significantly reducing the number of 
requirements levied by the Agency. 
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Tailoring process simplification - the law of -4^ 

unexpected consequences ). i 


Lesson Learned: A simplification in the tailoring process for various NASA NPDs and 
D&C Standards was to use the waiver & deviation process. Well understood, analogous 
process - easy, right? Problem was that waivers & deviations carry a very negative 
connotation in the HSF culture, and actually confounded the tailoring process rather 
than streamlining it. 

Recommendation: Tailoring is a good thing, and should not carry the stigma of some 
form of shortcoming that is rightly associated with deviations & waivers. Alter the 
tailoring process to remedy this negative implication. 
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Risk-Informed Design - risk as a commodity 





H i 


Orion 

Crew Vehicle 


Lesson Learned: Both Orion and the Lunar 
Lander initially struggled with meeting their 
mass constraints. Subsequently, those teams 
employed a design process that zero-based 
non-mission driven requirements, and then 
conducted risk-based trades to optimize 
vehicle reliability and safety. The result was a 
breakthrough, yielding a design solution that 
closed and was more reliable/safe than the 
initial approach that had "followed the rules." 

Recommendation: A rule-based approach can 
remove accountability for understanding the 
implications of design choices. Employ risk- 
informed design methodology up-front to keep 
the focus on mission success versus following 
"design rules". 
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Probability of LOC 


ISS LOC Trend 


O 

O 

LO 




Weight Scrub ERB CICP CPCB SBR DAC3 POD DAC3 Update DAC3 Update 

Results 1/23/2008 2/8/2008 5/2/2008 10/20/2008 11/19/2008 1 12/16/2008 2 3/24/2009 

12/10/2007 
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Projects Established Before Cx Program 



• Beneficial Effects 

- Early focus on projects allowed Cx to get out of the gate quickly. 

- Projects had to function absent formal integration early on, leading to an 
eventual Program Integration effort that was very lean by historical norms. 

• Detrimental Effects 

- SI Unit policy, Data Architecture, Facilities Management, & other integration 
strategies that were in a collective self-interest for the Cx Program were 
extraordinarily difficult to implement due to the cost of various contract 
changes. 

- Proliferation of boards & panels, duplicate yet subtlety different between 
projects, and then compounded upon Program stand-up, confounded the 
decision-making process. 

- Integrated analyses performed by Program uncovered technical issues 
(induced environments) that could have been caught earlier but for phasing. 
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Roles, responsibility, & authority - a non- 
thermodynamic application of entropy ) t \ 


Lesson Learned: RRA and decision-making were detrimentally effected by all 
three factors influencing Cx. This remained a challenge throughout the life of 
Cx and received diligent attention. Periodic 4D surveys indicated 
improvement overtime in response to corrective measures. 

Recommendation: There is no cookbook, no checklist solution for crisp & 
clear RRA in an Agency-wide, flagship program like Cx; likewise for a decision- 
making process that enables timely decisions at the appropriate levels to 
balance strategic vs. tactical viewpoints, with clearly delineated 
accountability. Rather, the lesson learned is to be diligent from the start & 
constantly pursue improvements in crispness and clarity. 
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A couple of instances of clarifying RRA... 



• Sometimes we had roles combined that needed to be separated, and 
sometimes we had roles separated that needed to be combined 

- The Cx Program struggled to establish an Integrated Master Schedule (IMS). 
When the role of IMS development was separated from IMS assessment, 
things quickly fell into place. 

• Recommendation: The IMS must be developed by the product/task providers & 
the integrators as an integration tool with that focus, not as a pull function from the 
assessors to an assessment tool. 

— Confusion existed on how the risk management process should be applied for 
cost threats. The risk management process was run by SMA, & the budget 
threat process by PP&C, leading to confusion & frustration. 

• Recommendation: Include Chief Engineers, Chief Safety Officers, and Resource 
Managers in the risk management process. Each has a specific role that should be 
defined in the Risk Management Plan. Caution is warranted when addressing 
issues that are driven by budget shortfalls. Review these issues in the project's 
planning/replanning arena before they are categorized as risks. Finally ; have 
budgets for risk mitigation because without funds, risks are automatically 
accepted and trend to issues. 
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Decision making - is only as efficient as 

RRAs are clear & understood ), \ 


• Lesson Learned: RRA and decision-making were detrimentally effected by 
all three factors influencing Cx. The Cx Program was spread across 
multiple Centers, multiple projects, & multiple integration entities, each 
with their own ways of vetting technical recommendations/decisions, 
sometimes leading to inconsistent management direction. Further, when 
an issue arose, there was often confusion about the appropriate "entry- 
point" into the decision making process. As with RAA, this remained a 
challenge throughout the life of Cx and received diligent attention, and 
periodic 4D surveys indicated improvement over time in response to 
corrective measures. 

• Recommendation: Invest the time & energy to define a comprehensive 
decision process that includes the Program, Technical Authority, Center, 
and Contractor decision processes. If you don't take the time to 
understand this up front, don't worry - it will reveal itself one impediment 
at a time. 
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Decision making - is only as efficient as RRAs 

are clear & understood ). \ 



Legend 


Level II Office 


Level III Office 
External 


Pending 
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Decision making - is only as efficier 
are clear & understood 


What you think is working like this: 



/ / / / 

MORS MCCS CxTF CxDS 
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as RRAs 



Is really working like this: 



* Multiply by # of Boards, Panels, Working Groups, Discipline 
Groups, Stakeholders, etc 
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Organization is organic - you'll never get it -4^ 

right, but you can make it better ). i 


Lesson Learned: The organization for a flagship program is organic and must 
adapt over time to the nature of the work. Size/scale & hierarchy can mask this. 
Cx was about a year slow in changing SE&I from a requirements focus to a design 
integration focus in the wake of SDR completion. However, that lesson was 
learned & not repeated; Orion & Ares implemented reorg's in the wake of PDR 
completions, making changes needed to progress to CDR, and Cx was in the 
process of a post-PDR reorg when proposed for cancellation. 

Recommendation: Revisit the organization at key milestone reviews to address 
the changing nature of the work ahead to achieve the next key milestone. This 
revisit should come in the form of a program plan update portion of milestone 
review up-brief to Agency management, since Agency management must be 
enlisted to mitigate social impacts across Centers due to ebb and flow of 
assigned work. 
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Organization is organic -you'll i 
right, but you can make it bette 
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Summary - Key Lessons Learned 



• Robust vs. optimal planning - the only certainty is that the funding will not match 
the plan 

• Schedule creep & the fixed base - the law of diminishing returns 

• In-house tasks - sustaining the NASA institutional base vs. affordably supporting 
the Programs - getting from "or" to "and" 

• Flight Tests - learning by doing 

• Communications among a far-flung team - interpersonal networks and IT 
applications can improve bandwidth 

• Tailoring of D&C Standards - drinking from a fire hose 

• Tailoring process simplification - the law of unexpected consequences 

• Risk-Informed Design - risk as a commodity 

• Roles, responsibility, & authority - a non-thermodynamic application of entropy 

• Decision making - is only as efficient as RRAs are clear & understood 

• Organization is organic - you'll never get it right, but you can make it better 
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Cx Lessons Learned 


• Program level 

- Cx Challenge 

- Final LL report 

— Individual office efforts 
— Cx LL wiki 

- KBRs 

- Multiple conference papers 

• Ares l-X 

- Wiki site 

- Knowledge Capture Report 

— KBRs including video 
interviews 

- Technical reports 
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• Ares 

- Ares I Final LL report 

- KBRs 

• Orion 

- KBRs 

- PA-1 Final Report 

• Ground Ops 

- 357 approved LLs being 
entered into the NEN LLDB 

- KBRs 

• Mission Ops 

- PM Challenge papers 

• Altair 

- Formal LL report 
— Engine KBR 
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SE&I Org 2007 vs 2010 
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Management Office: 
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Overview - Manifest 

FY08 President's Budget Baseline vs. PMR ' 07 




V = ISS Crew Flight 

= ISS Uncrewed Pressurized Cargo Flight 
U = Lunar Sortie Cooperative Crew Flight 
A = CEV LAS Pad Abort (PA) or Ascent Abort (AA) Test 
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V 

V 

V 

V 


Ares I / Orion Test Flight 

Ares I / Orion Crewed Test Flight 

Ares V Test Flight 

Ares V / LSAM Lunar Sortie Flight 


▼/ = Project DDT&E Complete / Schedule Reserve 

Deleted Flights and Tests 

Added Flights and Tests 
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Constellation Program Integration Roadmap - 12 months 






Program Manager's Recommend 

IOC Reserve Strategy ($M Cost) 



IOC Content Only 


5.000 

4.500 

4.000 

3.500 

(0 

| 3,000 
| 2,500 
.E 2,000 
** 1,500 

1.000 

500 

0 




FY06 

FY07 

FY08 

FY09 

FY10 

FY1 1 

FY12 

FY13 

FY14 

FY15 

FY16 




FY06 

FY07 

FY08 

FY09 

FY10 

FY1 1 

FY12 

FY13 

FY14 

FY15 

FY16 

Total 

|Total IOC Cost 


641 

1,915 

2,827 

3,122 

3,396 

4,497 

4,410 

3,964 

1,556 

659 

77 

27,064 

Orion & Ares DDTE 

378 

1,401 

2,024 

2,061 

1,687 

1,603 

1,215 

547 

111 

60 

60 

11,147 

DDTE 2 years sched Reserve 




14 

179 

458 

682 

1,056 

1,164 

547 

15 

4,115 

IOC Prod & Ops Proj Content 

263 

463 

676 

889 

1,258 

1,945 

1,982 

1,852 

18 

2 

2 

9,350 

IOC Prod & Ops Reserve 

0 

0 

72 

96 

203 

197 

194 

181 

1 

0 

0 

944 

IOC 65% Confid Level Reserve 

51 

55 

62 

69 

294 

337 

328 

262 

50 


1,508 


Cum Reserve as % Proj Budgets 7.20% 32.0% 
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Program "Trajectory" 


Risk Reduction Investments 

Dev. Testing, Eng. Dev. Units 


Procurement of Components 


Concept j Detailed Design 
Design 



30% 





Early investments 
effect Landing Zone 


SRR SDR We are here 
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ARC Task Distribution 



4 Crew Exploration Vehicle (CEV) 

■ LEAD Thermal Protection System (TPS) Advanced Development Project 

■ Aero/Aero-Thermal database development team 

■ Flight Software and CEV Guidance Navigation and Control support 

4 Crew Launch Vehicle (CLV) 

■ Expertise in Integrated Systems Health Monitoring (ISHM) including design and development phase 
health monitoring requirements analysis, CLV element fault detection algorithms development, and 
DDT&E and V&V tools development 

■ Support reliability assessment with Monte Carlo simulations 

■ Ascent Abort CFD Blast Analysis 

4 Mission Operations 

■ Exploration tools for flight controllers 

■ Development of new applications to future requirements for the Constellation training program 

■ Support DDT&E of multi-Center Command &Control systems, human-machine interaction 
requirements, mission control software development, project planning and management software 
systems, and documentation systems 

4 Level II 

■ PP&C - Data systems support 

■ SR&QA - Development of PRACA and SMA Information Systems 

■ SE&I - Support to Human Factors and Human Rating, Flight Performance, Thermal & ECLS, C3I, EVA 
and Ground/Mission OPS Systems Integration Groups (SIGs) 

■ T&V - Support to Human Factors and Human Rating, Integrated Thermal/ECLS, C3I T&V, and 
Avionics SW T&V SIGs 

■ APO -Support architecture refinement and conceptual design of future elements 
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DFRC Task Distribution 



♦ Crew Exploration Vehicle (CEV) 

■ Lead for Abort Flight Test Integration and operations 

■ Abort Test Booster (ATB) procurement 

■ Flight Test Article (FTA) and ATB integration 

■ FTA design, assembly, integration and test 

■ Independent Analysis and Oversight of prime contractors FTAs 

♦ Ground Operations 

■ Support preliminary definition and planning for CEV launch abort systems tests, drop tests, 
landing and recovery tests, flight re-entry and landing profiles, range safety requirements and 
integration 

♦ Level II 

■ SE&I - Support to C3I Systems Integration Group (SIG) 

■ T&V - Flight Test Planning Support 
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GRC Task Distribution 



♦ Crew Exploration Vehicle (CEV) 

■ Lead for Service Module (SM) and Spacecraft Adapter (SA) Integration 

- CEV Prime Contractor Oversight and Independent Analysis 

■ Flight Test Article and pathfinder production for SM and SA 

■ Integration Analysis and SE&I Support 

♦ Crew Launch Vehicle (CLV) 

■ Lead responsibilities for Upper Stage 

- Design and develop the Thrust Vector Control (TVC) subsystem 

- Design and develop the Electrical Power and Power Distribution System 

- Developmental Flight Instrumentation (DFI) Package 

- Leak detection Sensors Development 

- Purge System 

- Hazardous Gas Detection system 

■ Lead for Upper Stage module development for Advance Development Flight Test-0 (ADFT-0) 

■ Upper Stage SE&I support 

■ J-2X thermal/vacuum testing at Plum Brook B-2 facility 

■ Vehicle Integrated Design Analysis support 

♦ Level II 

■ SR&QA - Lead for FMEA Integration and program trends analysis; Support for document maintenance, 
program trending, integrated hazards analysis, and quality audits. Represent SR&QA at assigned Systems 
Integration Groups (SIGs) 

■ SE&I Integrated Power SIG co-lead, Book manager for several SRR IRDs, support to Analysis & Trades 
and Process & Tools Office, and support to Flight Performance, Thermal & ECLS, C3I and EVA SIGs 

■ T&V - Lead for Electrical Power Test Bed definition/architecture; T&V lead for Power SIG; Support to 
Avionics Test Architecture 

■ APO - Support architecture refinement and conceptual design of future elements 
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GSFC Task Distribution 



♦ CEV 

■ Communication and Tracking Support 

■ Mechanisms support 

♦ Level II 

■ SR&QA - Level II Software Safety and Assurance; Represent SR&QA at assigned Systems 
Integration Groups (SIGs) 

■ SE&I - Navigation SIG co-lead; Software and Avionics SIG co-lead, support to Flight 
Performance and C3I SIGs 

■ T&V - C3I T&V SIG lead, Navigation & Tracking T&V SIG lead, Book Manager for 
Software V&V Plan; Support to Avionics SW T&V SIG, RF link testing, and Interface 
verification requirements 

■ APO - Support architecture refinement and conceptual design of future elements 
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JPL Task Distribution 



♦ CEV 

■ Support to TPS Advanced Development Project 

♦ Mission Ops 

■ Lead for a Systems Engineering Process for OPS Development (SEPOD) - multi-center 
activity to lay out a road-map for the systems engineering processes related to ops 
development/preparation for the new program. 

♦ Level II 

■ SR&QA- Support for integrated hazards analysis and probabilistic risk assessment; 
Represent SR&QA at assigned Systems Integration Groups (SIGs) 

■ SE&I Software and Avionics SIG co-lead; support to Requirements, Analysis & Trades, and 
Process & Tools Offices; support to Nav & Track, Power, C3I, Human Factors and 
Ground/Mission OPS SIGs 

■ T&V - Support to Master Verification Plan, Cx Environmental Qualification and Acceptance 
Test Requirements (CEQATR), and Master T&V; Support to C3I T&V and Avionics SW T&V 
SIGs 

■ APO - Support architecture refinement and conceptual design of future elements 
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♦ Constellation Program 

■ Program Management 

• Manage and integrate Crew Exploration Vehicle (CEV), Crew launch Vehicle(CLV)/Heavy Lift Launch 
Vehicle (HLLV), Launch Systems, Mission Systems, Exploration Communication and Navigation 
Systems, and Extra Vehicular Activity (EVA) Projects 

♦ CEV 

■ Overall Project Management and Integration- CEV Prime Contract Management 

■ Lead for Crew Module and Vehicle Integration 

- CEV prime contractor oversight and independent analysis 

■ CEV government provided hardware 

■ Flight Test Execution 

♦ CLV 

■ Flight operations support to CLV including lead of First Stage Recovery System modification activities, 
Upper Stage RCS development and certification testing, and Abort Certification for all phases of CLV 
flight. 

■ Support Separation Certification for all phases of CLV flight, CLV reliability and safety assessments 
including launch site function, CLV Mission Operations Planning to the Operations Integration 
organization, and Avionics Simulation development 

♦ Mission Operations 

■ Project Management 

• Development of capabilities and planning for mission operations, crew training, and the Mission Control 
Center for ESMD human space flight missions. Will be coordinated closely with SOMD as they will be 
responsible for operation of these vehicles. 
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KSCTask Distribution 



♦ Ground Operations 

■ Project Management 

■ Responsible for achieving all Ground Operations objectives for the Agency allocated to the launch 
and landing sites. 

■ Leads DDT&E and logistics activities for all ground processing, launch and recovery systems 

■ Leads ground processing, launch and landing operations planning and execution 

♦ Crew Exploration Vehicle (CEV) 

■ Provides ground processing, launch operations and recovery support during DDTE phases of CEV 

■ GSE development support 

■ CEV prime contractor oversight and independent analysis 

♦ Crew Launch Vehicle (CLV) 

■ Provides ground processing, launch operations and recovery support during DDTE phases of CLV 
and CaLV 

■ Performs prime contractor insight and independent analysis 

■ Leads launch operations planning and execution for Advanced Development Test Flight (ADTF-0) 
and other flight demonstrations 

♦ Level II 

■ SR&QA- Support integrated hazards analysis and preliminary hazard analysis. Support Risk 
Management, quality assurance, and development of PRACA system. Represent SR&QA at 
assigned Systems Integration Groups (SIGs) 

■ SE&I - Supportability Systems Integration Group (SIG) co-lead and Ground/Mission Ops co-lead 

■ T&V - Support Integrated Element and end-to-end testing 

■ APO - Support architecture refinement and conceptual design of future elements 
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LaRCTask Distribution 



♦ Crew Exploration Vehicle (CEV) 

■ Lead for Launch Abort System (LAS) Integration 

- CEV prime contractor oversight and independent analysis 

■ Flight Test and Pathfinder Articles production for CM, LAS and separation rings 

■ Lead the CM Landing System Advanced Development Project 

■ Support the Thermal Protection System (TPS) Advanced Development Project 

■ Aero/Aerothermal, GN&C, Avionics S/W, and Displays & Controls support 

■ Independent Analysis and SE&I support 

♦ Crew Launch Vehicle (CLV) 

■ Lead Aerodynamic characterization of integrated launch vehicle stack, aerodynamic database 
development, and aeroelasticity test and analysis 

■ Lead for vehicle integration activities for Advanced Development Flight Test-0 (ADFT-0) 

■ Lead for CEV module development for ADFT-0 

■ Support Structural design and analysis, Guidance, Navigation, and Control (GN&C) development, 
Flight Mechanics and trajectory analyses 

■ Provide Systems Engineering support 

■ Support Upper Stage DDT&E 

♦ Level II 

■ SR&QA - Support for integrated hazards analysis and probabilistic risk assessment; Represent 
SR&QAat assigned Systems Integration Groups (SIGs) 

■ SE&I - Structures SIG co-lead and TPM lead; support to Requirements, Interface, Analysis & Trades 
and Process & Tools Offices; support to Software & Avionics, and Flight Performance SIGs 

■ T&V - T&V Flight Test and Performance Planning; Support Integrated Loads, Structures & Mech, C3I 
T&V, Avionics SW T&V SIGS; and Flight Test Planning support 

■ APO - Support architecture refinement and conceptual design of future elements 
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MSFC Task Distribution 



4 Crew Launch Vehicle (CLV)/CaLV 

■ Project Management for CLV and CaLV 

■ Responsible for achieving all CLV and CaLV objectives for the Agency. 

■ Lead associated Systems Engineering & Integration activities, all CLV & CaLV S&MA activities, and 
Upper Stage DDT&E 

■ First Stage design and Upper Stage Engine development contracts management, as well as leading 
or otherwise overseeing CLV associated demonstration testing. 

■ Responsibility for ADFT-0 and other flight demonstrations 
4 Crew Exploration Vehicle (CEV) 

■ Launch Abort Systems support 

■ Service Module support 

■ Abort Test Booster (ATB) requirements development and validation support 

4 Level II 

■ PP&C - Support to the Management Systems Office (MSO), includes Deputy Lead for Level II MSO 

■ SR&QA - Support integrated hazards analysis and probabilistic risk assessment; represent SR&QA at 
assigned Systems Integration Groups (SIGs); Support QA, risk management, and PRACA system 
development; Support Cx SR&QA panels. 

■ SE&I - Thermal & ECLS SIG co-lead, Environments SIG co-lead, Human Factors/Human Rating 
SIG co-lead, Loads & Structures co-lead; IRD & SRD Book Manager; support to Requirements, 
Interface and Process & Tools Offices, support to Software & Avionics, Flight Performance, Power, 
C3I, Supportability SIGs 

■ T&V - T&V Lead for Loads/Structures and Environments SIGs; Support to Flight Performance SIG, 
Human Factors & Human Rating SIG 

■ APO - Support architecture refinement and conceptual design of future elements 
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SSC Task Distribution 



♦ Crew Launch Vehicle (CLV) 

■ Serve in a focused program management and Integration role for Constellation Systems 
rocket propulsion testing 

■ Lead Sea Level Development, Certification & Acceptance Testing for Upper Stage Engine 
including facility modifications and test operations 

■ Support Altitude Development & Certification Testing for Upper Stage Engine 

■ Lead Sea Level Development Testing for Upper Stage Main Propulsion Test Article including 
facility modifications and test operations 

■ Lead Sea Level Acceptance Testing for Flight Upper Stage Assembly including facility 
modifications and test operations 

♦ Ground Operations 

■ Support Design, Development, Test & Evaluation (DDT&E) of propellant test and delivery 
systems, ground engine checkout facility simulation and analysis, engine and launch facility 
planning and development 

♦ Level II 

■ SE&I - Support to Flight Performance Systems Integration Group(SIG) (propulsion test 
integration), and Systems Engineering Processes and Tools 

■ T&V - Propulsion Test Integration and Coordination with Rocket Propulsion Test Management 
Board 

■ APO - Support architecture refinement and conceptual design of future elements 
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Orion Loss of Crew Loss of 
Mission 
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Orion Loss of Crew and Loss of Mission 
Analysis 


• As part of the Orion Integrated Design and Safety Analysis, Probabilistic 
Risk Assessment has played a very important role in increasing Vehicle 
safety and Mission Success. 

- To date, over 60 vehicle design changes have been implemented that improved 
LOC/LOM during five previous design cycles 

• OVEIWG Buy Back Round 1 (July 07 - Sept 07) 

• OVEIWG Buy Back Round 2 (Sept 07 - Nov 07) 

• DAC 2 LOC/LOM Improvements (Nov 07 - Oct 08) 

• Pre-DAC 3 LOM T Risk Reduction (Oct 08 - Nov 08) 

• DAC 3 LOC/LOM Risk Reduction (Nov 08 - June 09) 

- Completing the detailed reliability analysis early in the design has provided significant 
benefits 

• Have considerably reduced Orion's LOC/LOM risk 

• Also improved Orion's capability to mitigate failures by identifying potential operational 
workarounds 
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Orion's Risk Informed Design Process 


) 
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• Orion has employed a vigorous risk informed design process throughout the Project 
lifecycle 

• Probabilistic Risk Assessment (PRA) to estimate Loss of Crew (LOC)/Loss of Mission (LOM) 

• Hazard Analysis 

• Failure Mode Effects Analysis 

• Engineering and Operational judgment and analysis 

• To date, over 60 vehicle design changes have been made to improve LOC/LOM during 5 
design/analysis cycles 

- OVEIWG Buy Back Roundl (July 07 - Sept07) 

• Focused on LOC reduction 

- OVEIWG Buy Back Round 2 (Sept 07 - Nov 07) 

• Focused on LOM reduction 

- DAC 2 LOC/LOM Improvements (Nov 07 - Oct 08) 

- Pre-DAC 3 LOM T Risk Reduction (Oct 08 - Nov 08) 

• Focused on LOM T reduction 

- DAC 3 LOC/LOM Risk Reduction (Nov 08 - June 09) 

• Focused on reducing ISS LOC 

• These design changes resulted in the following LOC/LOM improvements 

- ISS LOC: ~l/52 — #531 

- ISS LOM: ~l/3 > 1/60 
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Summary of Orion Risk Influenced Design changes 
(OVEIWG, DAC-2, DAC-3 Assessments) 



• LRS: Redundant Forward Bay Cover Parachute, redundant NSIs for drogue chute mortars 

• MMOD: Buy backs for LS LOC (Option 2A) 

- Add an MMOD shield to the Avionics Ring ALAS extension protecting the heatshield (+15 lbs). 

- Thicken the PICA in the region of the heatshield that is susceptible to impacts from MMOD coming through the SM/CM separation gap (+18 lbs). 

- Add 12 layers of Kevlar and associated separation foam and attachment hardware to the cylindrical regions of the main propulsion tanks (+140 lbs). 

- Add 4 layers of Kevlar and associated separation foam and attachment hardware to the forward dome regions of the main propulsion tanks (+51 lbs). 

- Add 6 layers of Kevlar and associated separation foam and attachment hardware to the aft dome regions of the main propulsion tanks (+104 lbs). 

- Add a 0.15" thick composite panel inner wall to the umbilical assembly (+16 lbs). 

- Maintain the two wall configuration of the LIDS Adapter (if present) (0 lbs). 

- Increase the CM RCS nozzle wall thickness to 0.187" from 0.045" (+136 lbs). 

- Thicken the backshell TPS substrate facesheets to 0.037" in windward regions and to 0.025" or 0.030" in the leeward regions (+205 lbs). 

- Thicken certain 34" prop lines in SM by approximately 0.010" (+2 lbs). 

• Avionics/EPS: Addition of 2nd VMC, 2nd SCP, dual power feeds for VMCs & BFCS and ERM, data & power cross strapping, manual access to critical functions, Split Bus 
MBSU, addition of Ka-Band & S-Band HGA 

• Avionics: 3 rd VMC 

• GNC: 3 rd IMU, Addition of 2 nd VPU and 2 nd VNS external camera 

• Prop: Addition of selected CM and SM RCS Isolation Valves, 1FT OME TVC 

• Mechanisms: Redundant Solar Array Gimbals & Deployment motor windings 

• Avionics: FCM, DCM and CCM functions & RIU PEC functionality separated into individual power zones, RIU network function cross-strapped, development of BEC 

• ECLSS: 02 and N2 crossover valves added 

• LRS: Incorporated Option-1 re-design (removed NSIs, mortars, pilot chutes, FBC deployable mains, etc.) 

• ECLSS: Design (performance reasons) split ATCS from 2 to 4 loops (negative LOC/LOM impact), 0 2 System updates for LOM/LOC risk reduction 

• Avionics: Development of MRC, including 50% power mode (safe crew return) 

• Prop: SM valve update for Fie Press LOM risk reduction, CM RCS Thruster Flousing design for MMOD 

• EPS: Forward work (ERB actions) to mitigate SM Battery LOM risk (unpowered SM Sep) 

• ECLSS: ATCS Combined Fleat Exchanger, PP02 Sensor addition to CGA (non-vac CGA sensor), 02 & N2 Supply Regulators (series redundant in one leg), CM & SM 
ECLSS Drive Electronics (go forward with DAC-3 definition), CM Pump LOM Mitigation (cold spare pumps & controllers) 

• Avionics: Ability for DCM Function Re-host to FCM, CM & SM RIU Dual Cmd/Pwr backplane 

• Prop: SM Fie Pressurization change 

• EPS: ISS Docked Power Down Using MRC, SPTU Option (revisit in DAC-3) 

• MMOD: TPS Repair Capability, more SM Propulsion Line Protection 

- TPS (BS Repair): Current plan - NASA implement based on Shuttle experience 

• TPS - switch to Avcoat main heat shield 

• MMOD: 95% Tile Inspection effectiveness 

• MMOD: Added ~73lbs. mass to vehicle for ISS DRM (Back shell tile thickness) 
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CxP Integrated Assessment Study 

Summary Finding 4- External Contributing Factors 



The study team identified several external factors that contributed to the 
inability of the CxP to meet its cost and schedule commitments. Among 
those factors were: 

• An inadequate funding and reserve profile from the beginning of the 
program 

• Unplanned budget challenges and continuing resolution restrictions 

• Unfunded directed changes to design content 

• Lateness in establishing a program leadership team and its relative 
inexperience in development 

• Lack of early attention to systems engineering and integration 

• Agency strategy to design CxP systems to meet both the near term earth 
orbit objectives and the longer term lunar objectives 

Discussion of these and other external factors is outside the scope of this 
study, being addressed more fully in the "CxP Lessons Learned" briefing. 


Ref: “Constellation Program INTEGRATED ASSESSMENT STUDY”, Lee Norbraten, CxP Review Manager, 
January 28, 2011, slide 11. 
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